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In terms of Systems Space, Causality is one dimension. Causality distinguishes between: 

Teleoperation in which the operator’s actions affect the real world. Virtual Environments in which 
the operator’s action affects a 3D computer-generated simulated world. 

Other dimensions include : 

Sensory Modalities used by the System. 

Nature of the Models of Environment surrounding the user. 

Displacements or scaling in time or space between the user’s true position and the environment he or 
she interacts with. There are many systems - for example, the Head Mounted Display (HMD) - with 
the common theme of Technologically Medicated Experience, or Synthetic Experience. 

Examples of Synthetic Experience include : 

Virtual Reality - which uses a stereoscopic wide angle HMD to create the illusion of a SD 
surrounding fantasy world. 

Teleoperation - which uses devices such as an HMD and force - feedback handgrip electronically 
linked to a distant robot. The robot head turns to mimic the operator’s head motions and the robot 
arms mimic hand motion, so that the operator’s eyes and hands are effectively projected into the 
remote environment and the operator can look around and do things via the robot. The remote 
environment may be a human body. 

Other examples include : Microteleoperation, Sensory Prostheses, Telecommunication, and Synthetic 
sense. 

In this paper, I will deal with two types of Synthetic Experience - both in the medical sphere: 
Telemedicine, Virtual Reality, and Surgery. 
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> Teleoperation - Operator’s actions affect the real world 

> Virtual Environments - Operator’s actions affect only a simulated (computer) world 

> Technologically Mediated Experience (TMD) 

> General Model 
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Examples of Time (Synthetic Experience) 

• VR 

- Uses a stereo wide-angle HMD 

- Creates illusion of a 3D surrounding fantasy world 

• Flight Simulation 

- Accurately models the behavior of a real a/c 

• Teleoperation 

- Uses devices such as HMD and force-feedback handgrip 

- Linked to a distant robot body with robot arm + video cameras 

- Robot head mimics operator’s head motions 

- Robot arm mimics hand motions 


Microteleoperation 

• Uses a microscope and micromanipulator 

• Gives operator a sense of presence 

• Gives operator ability to act in a microscopic environment 

• Examples: 

- Retinal surgery / neurosurgery 

- Telecommunications 

- Sensory prostheses 

- Augmented reality 
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In This Paper 


• Two Types of Synthetic Experience dealt with 

- Virtual Reality and Surgery 

- Telemedicine 


y V.R. Systems 

• Allows user to enter a computer-generated 3D environment 

• Education, communication, military, entertainment, and medicine 

• Basic Components 

- Computer -f Software 

- Head-coupled display 

- Input devices (handgrip) 

y Software 

• Description of structure, appearance, and behavior of virtual world 

• System monitors input device (from user) and reforms model of the world 

• Systems sends f.b. to user - sound, tactile, force new state of virtual world 
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Geometric Models 

• 3D structures represented by 

- Surface Geometry (polygon patchwork) 

- Volumetric Data (CT or MRI data) 

- Finite Elements 


> Force Feedback 

• Tactile - cutaneous or force to hand or whole body 

• Multiple Joint Reaction Forces 


> Physiological Sensors 

• Pneumatic mouth-actuated controllers 

• Eye trackers 

• Dispersed switches 


► HMDs 

• User wears an HMD to experience sights and sounds of the virtual world 

> Input Devices 

• Joysticks, keyboards, buttons, and knobs 

• Allows user to manipulate virtual objects 

• Dataglove measures finger flexion 

• Magnetic tracking sensor transmits whole hand motion — ► virtual hand 
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► Surgical Applications 

• VR used in teaching, planning, training, and prediction of outcomes 

> VR Surgical Training 

• Allows student to enter and tour the body; e.g., can see how a synapse works 

• Allows two or more surgeons to interact with a virtual cadaver 

• 3D surgical simulator requires a detailed model of the body 

• Zettger’s kinematic model of the human skeleton 


i Virtual Cadaver 

• Central component of a VR anatomy trainer 

• 3D model of the anatomy plus sim. of biomechanics, physiology, and pathology 

• Stored library of congenital anomalies 

• Student can "see through” the skin to underlying structures 

> VR Surgical Simulation 

• VIRTUAL abdomen has been developed 

• Student can move from esophagus to rectum (cf. endoscopy) 

• Touching the mucosa could produce 

- A histologic view of mucosa 

- A videotape of gastric montility etc. 
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► VR Surgical Simulation (cont’d) 

• Can rotate organs to show hidden nerves, arteries, etc. 

• Techniques of ligation, isolation, etc., can be detailed 

t Present VR Simulation 

• Torso + stomach, duodenum, liver, bile ducts, gall bladder, pancreas, and colon 

• Instruments (scalpels, clamps, etc.) are provided 

> Five Requirements for Realistic Sim . 

• Fidelity 

• Object properties 

• Reality 

• Interactivity 

• Sensory input (force feedback, tactile, and pressure felt by surgeon) 


y VR Surgical Simulation (cont’d) 

• Present simulators use the HMD and Dataglove 

• Surgeons do not use a helmet or glove 

• Will need a telepresence surgery system with a 3D monitor in place of the HMD 

• Will need handles of actual instruments in place of Dataglove 
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► Telesurgery 

• NASA Space Station 


• Problems with time delay in data transmission 

• Problems with telepresence errors 

• Simulation and Training will be the main applications of VR 


VR Surgical Trainer 

• Three Basic Components 

- Physical model 

- The interface 

- The computer 

• Physical Model 

- Math, representation of patient + surgeon’s tools 

• Interface 

- Dataglove 

- Allows user to manipulate tools and patient model 

• Computer 

- Software + hardware 

- Runs model in real time 

- Surgeon practices surgery in the VE 
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t Abdominal Surgery Pilot Study 

• Dr. Joe Rosen, Dartmouth College 

• GOAL - Substitute for animals or cadavers 

• Includes a 3D model of bowel and surrounds 

• Bowel modeled as a linked set of rigid objects 

• Object deformation modeled by relative segment motion 

• Structures modeled - Surgeon's L. and R. Hands 

- Bowel Segments 

- Suturing Clamp 

- Needle /Sutures 

- Surgical Table, etc. 

• Audio feedback as analog to touch 


t Advanced Abdominal Simulator 

• Wider range of organs 

- Gall Bladder 

- Stomach 

- Bowel 

• "Fly Through” and "See Through” capability 

• Example: Placement of purse-string sutures within the four layers of bowel 

t Knowledge Gained to Date 

• Audio feedback not a good substitute for touch 

• Realism limited in bowel model 

- Interactivity of the tools 

- Surgeon / virtual op. room interface 

- Need to sim. physical reactions of bowel; e.g., bleeding 

- Lack of force f.b. in Dataglove 
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► Future Developments in Surgical Simulation Hybrids 

• Combination of VR simulator with real apparatus 

• Trainee holds and manipulates real devices 

• H igh- accuracy tactile f. b. 

• Good finite model of soft tissues in abdomen 

► Conclusions 

• VR provides a unique advantage for 

- Teaching 

- Training 

- Planning 

- Performance evaluation 

- Prediction of surgical outcomes 

• Realism limited in bowel model 

- Virtual patient 

- Surgeon’s tools 

- Operating room 


Examples of Telemedicine 

• NASA / Armenian (Yerevan) Satellite Link-1990 

• Post-Armenian earthquake disaster 

• Two satellites used - Intel and Comsat 

• Network 

- Yerevan General Hospital 

- Moscow 

- Satellites 

- Roaring Bend, Pennsylvania 

- NASA HQ 

- Baltimore Training Center 

- USUHS, Bethesda 

- L.D.S. Trauma Center, Salt Lake City 

- Houston Trauma Center 
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> Total Earthquake Casualties 

• Fatalities - 25,000 

• Injured - 25,000 

t Public Health Problems 

• Lack of disposable needles - population inoculation slow 

- Needles shipped in from Europe and U.S. 

• Lack of dialysis units* 

- Only eight old units in whole Republic 

- Modern units shipped in from Europe and U.S. 

♦Many cases of renal failure from crush injuries. 


Telemedicine Spacebridge - Communication Capabilities 

• Telecommunications equipment installed at four U.S. medical centers 

• Ground station at Republic Diagnostic Center, Yerevan, A.S.S.R. 

• Capabilities 

Real Time 

- Audiovisual 

- Fax 

- Broadcast Monday — Friday, 0900 — 1300 EDT (1800 — 2200) Armenian time 

- Slow Scan Video 

Non Real Time 

- Two portable field camcorders 
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Telemedicine Spacebridge - Communication Capabilities (cont’d) 

• U.S.S.R. accepts this offer and the satellite capability becomes operational 
July 5, 1989 

• Spacebridge operations terminated July 28, 1989 


Telemedicine Spacebridge - Operations 

• Daily agenda of topics established with one medical center taking the lead 

• On Fridays, Armenia faxes to the U.S. medical centers information on patients to 
be presented the following week 

• On Mondays, U.S. medical centers fax professional material to Armenia 

• Cases are presented by Armenian physicians (2/3 patients directly related to 
earthquake; 1/3 miscellaneous difficult management problems) 

- Patient sometimes present 

- X rays, CT, imaging presented 

- General discussion 

• U.S. physicians discuss cases with Armenian physicians and offer 
recommendations 

• Spacebridge extended through July 28, 1989, to accommodate consultations for 
burn victims of the Ufa train accident 
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Telemedicine Spacebridge - Results 


• Excellent results with communication equipment 

• Medical consultants developed efficient medical data acquisition and transmission 
procedures 

• Patient care/problems discussed 

- Public health 

- Post-traumatic stress 

- Infectious diseases 

- Epidemiology 

- Dialysis 

- Orthopedics 

- Prosthetics and rehabilitation 

- Imaging/lab 

- Burn management 

- Spinal cord injury 

- Plastic surgery 

- Vascular surgery 

- Eye injury 

- Aggravation of preexisting disease 


t Performance 

• Via consultations and advice of U.S. experts 

• Diagnosis and/or therapy in case of 250 American casualties was significantly 
changed 

• Over period of 3 months 

> UFA Train Disaster ( near Moscow) 

• 400 severe burn cases 

• Link extended to deal with this disaster 

• Link extended to burn center in Galveston 

► Post-link Follow-up 

• Several conferences - in U.S. , Russia, and Armenia have been held 

• Improvements in data transmission capabilities planned 
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